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Executive Summary 


In the fiscal year 2019, the Smithsonian Institution (SI) worked in collaboration with Affiliate museum 
educators at three Texas sites (City of Austin Parks and Recreation, Irving Arts Center, and Institute of 
Texan Cultures) to develop and disseminate instructional materials highlighting Latino and Asian Pacific 
American history, art, and culture for Texas’s high school Ethnic Studies courses. These instructional 
materials were disseminated to local social studies supervisors and curriculum specialists. The project 
focused on creating Smithsonian Learning Lab (SLL) collections to integrate museum-based Latino and 
Asian Pacific American instructional materials into these courses and to understand their usefulness in 
teaching. 


Christina Talbert, an independent evaluator, was hired to conduct a series of in-depth telephone 
interviews with 10 participants of the 2019 workshops. Participants were asked about their experience at 
the workshop, using the SLL themselves and in collaboration with teachers, and any barriers and lessons 
learned throughout the year regarding the SLL. 


Primary Conclusions 


The main conclusions of this study were: 


Curricula developers who participated in professional development workshops one year earlier and 
the teachers to whom they presented the Smithsonian Learning Lab valued the Lab mostly for the 
variety of authoritative and vetted Smithsonian digital resources. They envisioned the potentiality 
of the use of the Lab to engage students in inquiry-based learning. 


One of the main components of an effective professional development workshop is to include ample 
time to practice using the Lab. Our audience welcomed continuous support. 


1) 


2) 


3) 


Participants envisioned using the Smithsonian Learning Lab in a variety of ways. Over half of the 
participants described that they envisioned the SLL as a resource for student use. The following 
were some ideas for student use of the SLL: 


Envisioned Student Use of the SLL 


As a way for students to share their own learning 

As a way for students to create dialogue between their peers 

As way for students to conduct document-based question writing 

“As a way to move more towards the student-centered approach opposed to direct teaching.” 


“Like a resource for my students... what I like about it is the collections that are 
curated by museum professionals that are on there and those resources, those 
photographs, those collections. Those are really helpful for my kids.” 


Participants primarily used the SLL to develop curriculum for use based on student-centered 
approaches. They pointed out that the use can foster sharing and inquiries. Outside of curriculum 
development, participants used the SLL to find resources for content development and/or for 
personal research. Participants found the SLL to be effective for a variety of reasons, primarily 
being the abundance of vetted and reputable resources and/or the ability to use these resources to 
create content. Participants had not yet used the SLL in classrooms. 


“Tt is something that’s pretty much always within reach for me when a teacher says I 
need something or if we're designing something together, I’m always going to swing 
by the Learning Lab to find what's available to use. I use it a lot.” 


Participants described introducing the SLL to teachers through professional learning 
communities, professional development sessions, other types of meetings, and/or organic 
conversations. Participants did not know if the teachers whom they introduced to the SLL had 
used it in the classroom with their students. They did however recall that teachers expressed an 
interest in doing so and used a range of positive terms to describe the SLL. Teachers expressed 
that the SLL is effective because of the abundance of reliable resources, the accessibility of the 
website for students, and the usability of tools, such as hotspots, discussion questions, and 
annotations. 


“Whenever I have conversations with teachers looking for how to find an image that 
would be something kids could engage with in the classroom, I always mention the 
Learning Lab... because of the reputation and the fact that there’s so many, 1000s, of 
digital images that the kids would benefit from that they may never see in person.” 


4) 


5) 


6) 


7) 


8) 


Participants had mixed opinions about teachers using the SLL to align content to scope and 
sequence. The majority of participants agreed that the SLL can be used to align content through a 
connection to standards and/or the use of primary resources in creating content. The rest of the 
participants, while not expressing that the SLL could not be used to align content to scope and 
sequence, pointed out that the platform is not a necessary tool to do so. One participant explained 
that the SLL does not contain Texas-specific standards, and is thus unhelpful in aligning content 
to standards. 


“Tt allows them [teachers/curriculum specialists] to look at a standard and consider 
all of the possible way that’s a standard can be taught.” 


Participants suggested finding ways to introduce the SLL to more teachers in order to encourage 
them to create effective collections. Some suggestions for this were inviting teachers to 
workshops, providing virtual training via YouTube, sending out a listserv to teachers who are 
interested, attending state-wide conferences, and promoting the SLL through social media. 


“T think it starts with the workshop.” 


All of the participants were interested in joining an occasional meeting of a local support group 
provided by affiliate museums and a listserv to share and receive content regarding the SLL. 
Participants also expressed interest in additional virtual support. Some suggestions for virtual 
support included live video conferences, calls with teams of teachers/curriculum specialists who 
are interested, and/or recorded online training. Some suggestions for the content of virtual support 
include providing information about SLL updates, introductions to the SLL website, explanations 
of particular collections, and/or how to create content. 


“Maybe if [SI] recorded it like, this is how you make this lab or this is a conversation 
about this and here are some decks already made - anything that’s recorded and 
could be shared with teachers.” 


Participants expressed a couple of ways that the workshop best prepared them to use the SLL, 
primarily being the thorough introduction to the SLL and time allotted for collection creation. 
Participants also mentioned how effective Ashley and Philippa were at both introducing the SLL 
during the workshop and keeping in touch after. 


“That ability to go through with it, do it, build it, share it — I was sold. That time to 
dig around and build and look at collections and I immediately started talking to 
people back on campuses. The actual interaction with the lab.” 


Participants explained that the workshops could primarily be improved by allotting more time to 
introducing and using the SLL to create content, with some expressing that this portion of the 
workshop was “rushed.” As for future workshops with curriculum specialists and teachers, 
participants suggested providing breakout sessions for both types of participants to discuss their 
future use of the SLL. 


“IT would ’ve liked more time to do some of the work. I felt like touring all of the 
centers was important, but at times I felt maybe it was repetitive and while it was 
quality, it wasn’t what I thought I was signing up for.” 


9) Participants suggested the following improvements to the SLL website: 


Suggested Improvements to the SLL Website 


Adding PowerPoint capability to the collections 


Adding an ability to “level” (i.e. change the reading level to the intended grade level) historical 
readings based on grade level 


Adding video upload capability for students who are uncomfortable typing out answers 


Creating a variety of lesson plans developed by museum educators for teacher use 


Background & Context 


In the fiscal year 2019, the Smithsonian Institution (SD) worked in collaboration with Affiliate museum 
educators at three Texas sites (City of Austin Parks and Recreation, Irving Arts Center, and Institute of 
Texan Cultures) to develop and disseminate instructional materials highlighting Latino and Asian-Pacific 
American history, art, and culture for Texas’s high school Ethnic Studies courses. These instructional 
materials were developed by local social studies supervisors and curriculum specialists. The project 
focused on creating Smithsonian Learning Lab (SLL) collections to integrate museum-based Latino and 
Asian Pacific American instructional materials into these courses and to understand their usefulness in 
teaching. Participants used the SLL within their school districts during the academic year 2019/2020. In 
order to better understand how participants used the SLL and what lessons they learned from these 
experiences, SI partnered with Christina Talbert to provide evaluation support. 


Project Goals 


1. Inform the Smithsonian about how participants use the Smithsonian Learning Lab for curriculum 
development after participating in professional development workshops 

2. Provide insight into how participants shared the Smithsonian Learning Lab with teachers in their 
district, and retrospectively, how teachers then used the Smithsonian Learning Lab with students 

3. Improve future workshops 


Methodology 


In order to better understand how participants used the SLL in their districts, affiliate museum educators 
and Smithsonian staff members chose ten curriculum specialists and/or teachers that attended a workshop 
to contact and request interviews with. Participants were asked a variety of questions about their use of 
the SLL. Interviews lasted between 30-60 minutes. Prior to the interviews, participants were sent the 
interview script to review. 


Evaluation Questions 


1. How do participants use the Smithsonian Learning Lab in their school district? What are the 
lessons they learned by using it throughout the school year 2019/2020? 

2. How did participants introduce the Smithsonian Learning Lab to teachers in their district? What 
did teachers think about the Smithsonian Learning Lab? How have/will they use it? 

3. How could the Smithsonian improve future workshops? 


Sample 


A total of nine participants of the following job titles participated in the interviews: four curriculum 
coordinators (k-12, social studies), one curriculum coordinator (PreK-12, social studies), one assistant 
director of humanities (3-12, curriculum and assessments), one curriculum support specialist (6-12, social 
studies), one facilitator (6-12, social studies), and one US history teacher (11 grade). 


Findings 


This section includes findings from interviews with nine workshop participants. Findings are in 
the following order: 


Envisioning the Use of the Smithsonian Learning Lab................. cece ec eceec ee eeeeeec eee eeeeneaeees 6 
Motivation to Use the Smithsonian Learning Lab............ 0.0... cece ccc e eee e eee ee ence eens enae teenie en 8 
Curriculum Specialist Use of the Smithsonian Learning Lab ................. cece eeeee ence eee e eee enees 8 

Curriculum Specialists - Effectiveness of the Smithsonian Learning Lab........................0000 10 
Teacher Use of the Smithsonian Learning Lab..............c cece eccce ence cece ene eee e cent nae ea eens eenaes 11 
Teachers - Effectiveness of the Smithsonian Learning Lab.............. 0.0... cece cece eee eect eee e ees 13 
Use of the Smithsonian Learning Lab in the Classroom.............. 0... cceeeee ence nee neeeeene eens anes 14 
Aligning Smithsonian Learning Lab Content to Scope and Sequence................e cece eee ene ee 15 
Suggestions for Encouraging Teachers to Create Effective Lessons................cceeceeeene ene ee eee 16 
Suggested Improvements to the Smithsonian Learning Lab................. cece ce ec ee ee ne ene ene ee ees 18 
Suggested Ideas for Additional Smithsonian Suppotrt............... 0c ce cece ne ee eee ee eeee teeta eenees 21 
Best Aspects of the Workshops................ccecceeeeeeee cece eee ee ee eases ene ene ene eee eae eae eae eeenseeees 22 
Suggested Workshop Improvements................ sce c cece scene ence eee ence eeeeeeeae ee eeeeneeneeaenaeaes 22 


Envisioning the Use of the Smithsonian Learning Lab 


Curriculum specialists envisioned using the Smithsonian Learning Lab in a variety of ways. Over half of 
the participants described that they envisioned the SLL as a resource for student use. The following were 
some ideas for student use of the SLL: 


- asa way for students to share their own learning 

- as away for students to create dialogue between their peers 

- as away for students to conduct document-based question writing 

- “as away to move more towards the student-centered approach opposed to direct teaching.” 


A couple of participants mentioned using the SLL’s searchability features to find resources, with 
participants describing these resources as high-quality and vetted (Figure 2). Other participants envision 
the SLL as a place to create materials for the classroom. For example, the assistant director of humanities 
said s/he envisions it “as a resource for teachers to curate and share materials with students,” while the 
social studies facilitator said, “I envision this as a way for me to design really focused collections or 
resources that teachers can use.” A curriculum coordinator added, “teachers spend so much time lecturing 
so this is a great opportunity for them to use pictures and images for activities.” Lastly, the curriculum 
support specialist envisioned the SLL to be used “as a presenting tool for students to create something and 
present what they know using evidence.” 
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Figure 2. Example of the search engine for Resources on the Smithsonian Learning Lab 


Motivation to Use the Smithsonian Learning Lab 


A variety of participants explained that the reputable 


resources are what motivates them to use the Participants turn to the Smithsonian 
Smithsonian Learning Lab, including some Learning Lab because of its variety and 
curriculum coordinators, the assistant director of abundance of trusted resources 


humanities, the social studies facilitator, and the US 
history teacher. These participants turn to the SLL 
because of its variety and abundance of trusted resources. For example, a curriculum coordinator said, 
“whenever I have conversations with teachers looking for how to find an image that would be something 
kids could engage with in the classroom, I always mention the Learning Lab again because of the 
reputation and the fact that there’s so many, 1000s, of digital images that the kids would benefit from that 
they may never see in person.” A couple of the participants expressed that they are motivated to use the 
SLL because of the ability to create content. The social studies facilitator said, “I can look at 500,000 
sources and pick them and organize them there, kick it out, and share it.” One curriculum coordinator said 
that the SLL is useful in categorizing his/her work through collections, stating “I have a couple of projects 
going on at one time and it’s beneficial to have them in separate pieces and bundle...for me, it can be 
bundled differently and categorized.” Other participants explained that they are motivated to use the SLL 
because of the student usability aspect of the website. For example, the social studies facilitator said, 
“we’re also really interested in our students doing creations to show their own learning,” and a curriculum 
coordinator said, “you have multiple styles of learning — no child or adult learns in the same manner so 
here’s another tool that can be used to reach different learners.” Lastly, one curriculum coordinator said 
s/he is motivated to use the SLL “as another tool to drive discussions with students.” 


Curriculum Specialists Use of the Smithsonian Learning Lab 


None of the participants had used the SLL in the classroom themselves, either because they do not usually 
spend time in the classroom teaching students or because they did not have a chance to this year due to 
school closures during the COVID-19 outbreak. However, many participants mentioned using the SLL to 
develop curriculum for teachers to use and/or share with students. Three participants, a social studies 
curriculum support specialist, social studies facilitator, and a curriculum coordinator, had developed 


10 


collections for student use. The social studies facilitator described one of his/her collections as a Civil 
Rights collection that connected to a Texas-specific standard and the other as a student-activity where 
students were prompted to think about personal history and their experience during the current time 
period — “personally, nationally, and in the world.” This participant discussed how helpful the Harvard 
Zero Thinking Routines were in creating lesson plans, explaining that “they’re awesome. It’s so easy to 
drop them in and transition. I did an art analysis collection for the conference with the see, think, wonder 
card and then linked it to archives” (Figure 3). This participant also explained that s/he uses the SLL 
to find resources, saying “it is something that’s pretty much always within reach for me when a teacher 
says I need something or if we’re designing something together, I’m always going to swing by the 
Learning Lab to find what’s available to use. I use it a lot.” The curriculum coordinator that had 
created collections for student use described a variety of collections s/he created for world culture and US 
history regarding religion, reconstruction, and communism. The social studies curriculum support 
specialist built a collection about Native American Code Talkers in World War 1 to provide instruction 
for a specific Texas standard “that says students will be able to explain various reasons that led to Allied 
success.” This participant explained that s/he “created a collection of visuals and primary sources and 
secondary readings and they [the students] analyzed the documents and visuals to try to come to that 
target that identified that code talker operation as an element of allied success. When I use your resources, 
I think of a lot of analysis strategies such as visual analysis. We ask our students to collect evidence for 
their answers and that’s what the Learning Lab can do for them. It can help them collect evidence in the 
Lab and base an opinion and defend their answer with evidence.” Two participants said the only content 
they developed on the SLL was the collections they created during the workshop, with one describing 
his/her collection as “some resources around Asian American history.” The other participant explained 
that s/he used the collection that s/he created during the workshop as an example for introducing the SLL 
to teachers, but did not explain the content of this collection. One curriculum coordinator developed a 
collection primarily to introduce the SLL to teachers during a professional development session. S/he 
described this experience, stating, “I modeled a strategy for teachers called “Which one doesn’t belong” 
and I went through a couple of curated artworks from different times and asked teachers which of the 
images doesn’t belong and why and we had individual discussions and a whole group discussion.” 


What do you see? 


What do you think about that? 


What does it make you 
wonder? 


Figure 3. Example of a Harvard Zero Thinking Routine slide on the Smithsonian Learning Lab 


Just over half of the participants had used the Smithsonian Learning Lab for something other than 
developing curriculum. Two of these participants described using the SLL to find resources for 
curriculum development and/or encouraging teachers to do so. The three other participants had used the 
SLL for personal research, such as creating collections for a personal genealogy club. One of these 
participants said that s/he finds resources that s/he personally wants to keep. The same participant 
discussed the possibility of using the SLL for a history fair so that students can “do research for an 
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original think tank to share with everyone else what they’re thinking about for a historical product,” and 
explained how the SLL is perfect for this because students will find primary resources and if their project 
changes over time, they can simply continue building it on the SLL. 


Curriculum Specialists - Effectiveness of the Smithsonian Learning Lab 


When asked what participants found particularly effective about the Smithsonian Learning Lab, most of 
them mentioned the resources. They described resources as vetted and reputable, as they are available 
through the Smithsonian. A few participants described the ability to curate content on the SLL as 
particularly effective. For example, the assistant director of humanities said, “it’s interesting to go through 
the collections - to make choices and figure out how you might curate something. It’s a good experience 
to look at an image or document or piece of art to find out what questions I can ask about it.” A 
curriculum coordinator said, “if you’re designing a learning experience that you want to evoke a certain 
emotion or students to respond in a certain way — imagery is everything so there’s a lot of value to using 
this tool.” When describing the effectiveness of the website itself, participants mentioned both 
searchability and shareability (Figures 4 and 5). For example, the social studies facilitator said, “TI like that 
you can search by everything everywhere and look at collections people have created and the filter 
features are super helpful,” while the curriculum support specialist said, “the shareability. I know I can 
create a collection and share it with a link easily and give it to anybody.” 


Share Learning Lab Collection 
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Figure 4. Example of share features on the Smithsonian Learning Lab 
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Age Range 
Preschool (0 to 4 years old) (1) 
Primary (5 to 8 years old) (6) 
Elementary (9 to 12 years old) (21) 
Middle School (13 to 15 years old) 
(52) 
Post-Secondary (26) 
Adults (24) 


High School (16 to 18 years old) 
(62) 
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Figure 5. Example of the search engine for Learning Lab Collections on the Smithsonian Learning Lab 


Teacher Use of the Smithsonian Learning Lab 


Participants described introducing the Smithsonian Learning Lab with both teachers and other curriculum 
specialists in a variety of ways, with many mentioning multiple ways they have done so. Almost all of the 
participants had introduced the SLL to other specialists and/or teachers through meetings, dedicated 
Professional Learning Communities (PLC’s), and/or professional development (PD) sessions. Participants 
of each job title had shared the SLL to colleagues (i.e. some coordinators, the US history teacher, the 
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social studies facilitator, the curriculum support specialist, and the assistant director of humanities). When 
asked how they introduced the SLL during these meetings/PLC’s/PD sessions, many participants said 
they described the SLL as a reputable website with an abundance and variety of primary resources for 
students to engage with. Others introduced the SLL as a website to engage students using Harvard Zero 
Thinking Routines or prompted discussions (Figures 3 and 6). A curriculum coordinator described the 
SLL as “a resource put on by the Smithsonian that will allow them [teachers] to annotate or utilize 
documents within the Smithsonian system.” Two participants introduced the SLL with teachers 
organically, with one describing the experience — “whenever I have conversations with teachers looking 
for how to find an image that would be something kids could engage with in the classroom, I always 
mention the Learning Lab again because of the reputation and the fact that there’s so many, 1000s, of 
digital images that the kids would benefit from that they may never see in person.” Two participants 
introduced the SLL to teachers simply by embedding it in their curriculum, such as online curriculum unit 
maps and/or curriculum documents. Some other ways participants have introduced the SLL to teachers 
include newsletters, email blasts, and tweets. Most of the participants did not explain how many teachers 
they discussed the SLL with or how many times they introduced it. Of the ones that did, some described 
discussing it with up to sixteen curriculum specialists and/or teachers with others describing up to 
seventy-five. 


Only one participant, the assistant director of humanities, did not share the SLL with teachers in his/her 
district. When asked why this was, s/he explained that his/her “role, because I don’t do a ton of 
interfacing with teachers... have coaches who interface with teachers and so it’s something that they talk 
about with PLC’s or share at meetings with teachers.” S/he went on to explain some barriers to sharing 
the SLL with teachers, such as how there are “so many things out there for people to use so how do you 
always keep it fresh and new and help people use it.” S/he explained further “[we] haven’t been with 
teachers at all since March, anything that we are sharing now is within context of preparing for a virtual 
learning environment or distance learning and less focused on ‘Here’s a resource to use’ even though 
that’s a great tech tool to collect feedback and help students collaborate and engage and build community 
in the classroom. So, components of what we need in pillars of distance learning are there. So, we should 
probably try to push this out more in the summer and help teachers embed this resource more into their 
classroom.” 


Questions for Discussion 


1. What do you see? 


2. What do you think about that? 


3. What does it make you wonder? 
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Figure 6. Example of discussion questions on the Smithsonian Learning Lab 


Participants recalled that teachers and other curriculum specialists had a positive view of the Smithsonian 
Learning Lab when it was introduced to them. The terminology that participants recalled hearing from 
curriculum specialists and/or teachers when they introduced the SLL to was the following... 
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Figure 1. Terminology participants remembered teachers and/or other curriculum specialists using to describe the 
Smithsonian Learning Lab 


Throughout the interview, participants stated how trustworthy the resources on the SLL are and that 
teachers and other curriculum specialists were interested in using the site because of the reputable 
resources they found. For example, the social studies facilitator said, “teachers appreciate that this is a 
safe place to go for teachers to dig and search and look at things instead of googling the Cold War.” 


Three curriculum coordinators offered information about how teachers reacted to creating collections on 
the SLL. One explained that when teachers realized they would need to create collections, they asked 
him/her to build them. Another described how “overwhelming” the “tremendous” amount of resources 
available on the SLL can be to teachers. Lastly, one explained that teachers were excited and wanted to 
know how to “incorporate this into the lessons we’ve already planned for the year.” 


Teachers - Effectiveness of the Smithsonian Learning Lab 


Participants were asked if teachers mentioned anything 
particularly effective about the Smithsonian Learning Lab Teachers fe ound the abundance 

and their answers were similar to that of the curriculum and reliability of resources on 
specialists that were interviewed. Over half of the the SLL as effective in creating 
participants explained that teachers found the abundance and — content for their classroom. 

reliability of resources on the SLL as effective in creating 

content for their classroom. One curriculum coordinator said, “I think they were surprised at the wealth of 
resources because teachers are getting bombarded with going to different sites for different things and I 
think they were blown away by how many resources they have access to and the quality of the images and 
pictures and ability to pull up intricate pieces of artwork and getting into seeing the details.” Participants 
recalled teachers saying that specific tools on the lab were effective in creating content, such as 
annotations, adding questions, and adding hotspots to resources (Figures 6, 7, and 8). Another effective 
aspect of the SLL that a curriculum coordinator mentioned teachers speaking about was the ability to use 
it with students, explaining “this is a place where teachers can call out and create a collection that’s based 
on a standard or topic they’re teaching and have this ready for students to use.” Another curriculum 
coordinator explained how easy it is to find content on the SLL, saying, “it wasn’t challenging to find 
specific resources. I felt like the search options — it was fairly easy to use, even teachers who don’t do 
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much online” (Figure 2). Lastly, one curriculum coordinator explained how easy it is to navigate on the 
learning lab website, saying “even the layout was user-friendly.” 


(TO READ LATER) Braceros, by Domingo Ulloa 


Date: 1960 


Domingo Ulloa painted this canvas after several visits to a 
Bracero camp in Holtville, California. The Bracero Program 
(1942-64) was a bi-national effort that brought Mexican guest 
workers, known as braceros, to fillin agricultural labor shortages 
caused by World War II, Ulloa’s crowd of workers, who peer 
dejectedly through a barbed-wire fence, reinforce the mounting 
public protest against their poor living and working conditions. 
His composition recalls photographs of concentration camp 
inmates, which Ulloa--a World War II veteran--was familiar with 
Ulloa later stated, "Most of my paintings are inspired by the 
common people in their work, in their joy, and their struggle” 


Figure 7. Example of an annotation on the Smithsonian Learning Lab (Annotations can also be inputted using 
hotspots and metadata) 


Q 
Bi shapes 


Highlighting areas of interest 


After watching the video in the previous 
tile, take a few minutes to consider the 
hotspots here. How might the artist use 
colors, shapes, lines, contrast, texture to 
convey meaning? 


How might you use hotspots in your 
classroom? 


Figure 8. Example of the hotspot tool on the Smithsonian Learning Lab 
Use of the Smithsonian Learning Lab in the Classroom 


None of the curriculum specialists had used the Smithsonian Learning Lab in a classroom. One 
curriculum coordinator said that s/he “offered to model it, but haven’t had the chance to do that.” 
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The majority of participants did not know if the 
teachers that they introduced the SLL to had used it in The majority of participants did not 
the classroom with their students. One curriculum : 

. know if the teachers that they 
coordinator stated that the curriculum specialists s/he : ; p . 
works with had created collections in the SLL, but that “7 oduced the S. mithsonian Learning 
they do not teach classes, saying “my curriculum Lab to had used it in the classroom 
specialist do not teach classes — they write and develop —_ With their students. 
curriculum so I know they’ve used them within 
curriculum documents for teachers to use.” Another curriculum coordinator said s/he knew that secondary 
and high school teachers in his/her district had created collections. When asked to describe these 
collections, she explained that some high school teachers created “quizzes and interactive pieces where 
the kids could scroll over images with descriptions,” and that “some elementary [teachers] started 
collections but I don’t know to what degree.” When asked if s/he knew how these collections were used 
in the classroom, s/he stated that teachers may have projected the images, saying “a teacher might have 
had just the image presented so that all the kids could see it on a large format because our kids don’t have 
individual devices so the easiest way to do that is projecting.” S/he then explained that s/he is not sure 
that students knew the images came from the SLL, so s/he is not aware of their reaction to the website. 
The US history teacher said that she had used the SLL in her classroom, but did not create the collections 
herself. Rather, s/he used “hyper-focused” collections that she found in the SLL created by museum 
professionals. S/he explained that the “collections are really helpful in bringing some of the content to life 
in a way because the kids can see those collections. Especially my population that doesn’t have access to 
a lot of museums so this brings the museum to them.” Lastly, one curriculum coordinator knew that the 
teachers s/he introduced to the SLL had used the lessons s/he created but did not “get feedback on how 
students responded to it.” 


Since participants did not know if the teachers they introduced to the SLL used it with students, they did 
not mention any barriers or benefits of using the SLL. Additionally, participants did not recall that 
teachers had any suggestions for improvements to the SLL, with one curriculum coordinator explaining 
that “feedback was very positive because it’s a good resource and it’s the Smithsonian.” The social 
studies facilitator did mention that the teachers s/he introduced the SLL to were confused about what 
happens to students quiz answers once they are filled in and that teachers wanted to interact with students 
in this manner but didn’t understand how to do so effectively. This participant went on to explain that 
“they’re [students] not filling things out for you to grade later. And sometimes I think that’s nice and 
other times it takes away from the purpose.” 


Aligning Smithsonian Learning Lab Content to Scope and Sequence 


Participants were asked their opinion about using the 


SLL to help teachers align content to their scope and Participants agreed that the Smithsonian 
sequence. Six of the participants agreed thatthe SLL [ earnin g Lab allows them to help 
allows them to help teachers align content to their teachers align content to their scope and 


scope and sequence either through a connection to : ; 
sequence either through a connection to 
standards or the use of primary resources in creating F 
standards or the use of primary 


content. Of the participants that agreed, they had a . : 

diverse pool of jobs, such as some curriculum POROUI CES ESIC COMICRE 
coordinators, the assistant director of humanities, the 
US history teacher, and the social studies facilitator. Regarding the connection to standards, the assistant 
director of humanities said, “it allows them [teachers] to look at a standard and consider all of the possible 
ways that a standard can be taught.” Regarding the abundance of primary resources used to create content, 
the social studies facilitator said the SLL helps align content because of the “vetted, quality resources for 
teachers and students that are organized well and easy to use and share and that everything is there.” 
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Three participants did not use the SLL to align content to scope and sequence for two reasons, though 
neither disagreed that the SLL could be used to align content. Two of these participants did not feel that it 
is necessary to use the SLL to align content to scope and sequence. For example, the social studies 
curriculum support specialist said “I don’t know if there’s a better way to organize it because I have 
everything aligned in my head,” while another participant, a curriculum coordinator said “alignment has 
to be there but I don’t know that the Learning Lab explicitly does that but it isn’t a hindrance...I don’t 
think it’s the Lab’s job to do it effectively. That’s not what - it’s designed to be more of a resource so 
teachers have the ability to create curriculum around it. It should come from a teacher or person in my 
position and not necessarily the Smithsonian.” One of these participants, a curriculum coordinator, did not 
think that the SLL linked to Texas-specific standards but that if “you could search for a TEKS (Texas 
Essential Knowledge Skills) [another term for standard], that would be very helpful but I haven’t found 
that feature” (Figure 9). 


Standards 


= Common Core State Standards (CCSS) Initiative: Mathematics Standards 
> 


Standards for Mathematical Practice 

> 

Counting and Cardinality 

5 

Operations and Algebraic Thinking 

5 

Number and Operations in Base Ten 

5 

Measurement and Data 

3 

Geometry 

> 

Number and Operations—Fractions 

: ? 

Ratios and Proportional Relationships 
Figure 9. Example of the list of standards on the Smithsonian Learning Lab. The following standards are available 
to search through and/or add to the description of collections: Common Core State Standards Initiative: 
Mathematics Standards, Common Core State Standards Initiative: English/Language Arts Standards, Next 
Generation Science Standards, College, Career, and Civic Life Framework for Social Studies State Standards. 
Some, if not all, of these standards, provide links to collections. 


Suggestions for Encouraging Teachers to Create Effective Lessons 


Participants suggested a range of ways the Smithsonian 
can encourage more teachers to create effective content Participants said that teachers cannot 


on the Smithsonian Learning Lab, primarily by understand how to create effective 
introducing it to more teachers. For instance, content without being introduced to the 
participants said that teachers cannot understand how Smithsonian Learning Lab itself. 


to create effective content without being introduced to 
the SLL itself. Some participants suggested providing 
more workshops like the one they attended, while others suggested connecting to teachers virtually, either 
through YouTube videos, a listserv, and/or social media. One curriculum coordinator said that attending 
state-wide conferences would introduce the SLL to many teachers, while the assistant director of 
humanities suggested “getting newer teachers to use it by connecting with teacher programs to make this 
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something that’s available so that newer teachers see it as something that’s part of their repertoire.” The 
assistant director of humanities and a curriculum coordinator suggested using the SLL as a way for 
teachers to showcase their work, such as “creating or giving an opportunity to create exemplar teacher 
lessons” or possibly creating a badging system similar to Google certified teachers to reward teachers for 
their efforts. Another idea was to encourage more teachers to educate other teachers about the SLL. For 
example, the assistant director of humanities suggested “if you’re able to identify high users and ask those 
teachers to campaign the work for you - because teachers hearing from other teachers works really well.” 
One curriculum coordinator said “I feel like we’re a good source for that - presenting it [the SLL] to us to 
roll out to teachers,” thus encouraging the Smithsonian to continue working with curriculum specialists. 
On the other hand, the social studies facilitator felt that it would be best for teachers to experience what 
s/he experienced by attending the workshop, saying “I’d be willing to bet that no one went back to it 
[after the PLC s/he held to introduce the SLL] but they were excited. ‘Why do you think no one went 
back to it?’ Because it was a short brief moment of their PLC, but I think if they had the experience in 
Irving that I had, then they’d go back and be using it. It was nice to have that. It was a great day to have 
away from creating and lesson planning. | think if they had that, then they’d be more likely to revisit it but 
I find that with resources in general because there are other things I’m sharing and I think a lot of it goes 
in an email folder and doesn’t always get interaction.” One curriculum coordinator suggested reaching out 
to curriculum service centers and working with their employees. The social studies facilitator mentioned 
that it is important to introduce the SLL as a source that can be used to align content to standards and that 
“because you have such a variety of resources, there’s always going to be a way to find something that 
aligns. In order for you to get teachers to build and submit.” 


As for what content is considered effective, more than half of the participants said that it must be aligned 
to standards. Other participants explained that effective content contains primary and vetted resources. 
For example, the social studies facilitator explained that “on the Learning Lab, all of your sources are 
cited. The history is there. There’s no question about where an artifact came from in a collection.” A 
couple of participants expressed that effective content created on the SLL would incorporate discussion 
components, writing components, and strategies (Figures 3, 6, and 10). 


During the discussion about effective content on the 


SLL, some participants expressed concern about using Some participants expressed concern 
particular SLL content created by teachers, since they about using particular Smithsonian 
do not know the quality of the content. For example, Learning Lab content created by 


one curriculum coordinator explained, “that’s one thing 
that I worry about to a degree because that’s something 
we struggle with as curriculum developers because we 
are an open resource. At one point we had a discussion 
about teachers submitting things to add to curriculum and we went in circles about how to monitor that to 
ensure high-quality resources and promote our mission and vision as that’s one thing that I worry about a 
little bit by just saying this resource is great and I don’t want to endorse collections I haven’t seen.” Other 
participants expressed this concern as well, with a curriculum coordinator saying “if I’m searching and 
using other peoples created lessons, then how am I going to know it’s good?” and another curriculum 
coordinator saying collections “could be good or bad considering if it aligns with how we teach social 
studies.” The curriculum support specialist expressed that providing an indicator that identifies collections 
created by SI staff prior to clicking into the collection would be beneficial for him/her (Figure 11). 


teachers, since they do not know the 
quality of the content. 
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Questions for Discussion 


1. Why might this topic matter to me? 


2 . Why might it matter to people around me (family, friends, city, 
nation)? 


3. Why might it matter to the world? 


Figure 10. Example of a writing component on the Smithsonian Learning Lab 


> 6 9 


“Hyphenated Americans”: ... 
[9 | 


Figure 11, Example of how a collection is viewed when using the search engine, without clicking into the collection. 
Each SI employee has the SI icon, the sunburst, next to their name, which is viewed when hovering the mouse over 
the collection. 


Suggested Improvements to the Smithsonian Learning Lab 


Participants explained that they use a large variety of online learning platforms. A few participants find 
News ELA effective because it is “a collection of recreated news stories that you can use for a variety of 
content areas like the Learning Lab in a sense that you curate your own collections and ask questions and 
annotate things but different because the content’s all the same, like it’s all articles.” ABC Cleo is an 
online learning platform a few of the participants also found effective, mentioning how reliable the 
content is, how it is easy to navigate, how it is updating daily, and how the site continuously seeks 
feedback from teachers to make changes. Another source a couple of teachers mentioned was the Howard 
Zinn education project. What participants liked about this online learning platform was that it pushes 
against “status quo history” and contains fully developed lesson plans. A couple of participants mentioned 
Google Arts and Culture, with one saying that it “has a lot of interaction and visual engagement,” and 
“allows you to tour [museums] and look at artifacts.” Two participants talked about using PBS 
LearningMedia, with one saying, s/he uses it because “they have videos — they are pretty good about 
having videos, maybe some imagery...and then creating lessons around it so they have a pretty good site 
with a ton of lessons.” Two participants explained their use of the Library of Congress website, with one 
saying that s/he uses it because it holds so many resources. Other online learning platforms that were 
mentioned include Discovery Ed, Gilbert Learning, The DBQ Project, Common Lit, the Texas Data 
Historical Society website, the Gale Database, the National Archives, the Institute for American History, 
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Britannica Online, National Geographic, Ed Puzzle, Flip Grid, and the Stanford History Education 
Group. 


When comparing the SLL to other online platforms, participants offered an array of compliments about 
the SLL. Some of these compliments were the following: 


- the SLL updates daily with new content 

- the SLL is useful when looking for fully developed educational experiences 

- the SLL is considered more grade-level appropriate for grades 6-12 than other platforms 

- the SLL is broad enough to find “anything that you’re looking for” 

- the SLL is more visually appealing and/or contains more visual content than other platforms 

- the SLL is “cleaner - the purpose of the learning lab is to give you freedom of creation - you 
can go in any direction” 


Participants also brainstormed the following improvements for the SLL website: 


- adding PowerPoint capability to the collections 

- adding an ability to “level” (i.e. change the reading level to the intended grade level) 
historical readings based on grade level 

- adding video upload capability for students who are uncomfortable typing out answers 

- creating a variety of lesson plans developed by museum educators for teacher use 


The assistant director of humanities mentioned two tools that may be useful to add to the SLL: Play Posit, 
a tool that pauses videos for students to answer questions and Stackup, a tool that measures how much 
students are reading and at what level. Some other suggested improvements regarded searchability and 
collection building. As for searchability, one curriculum coordinator expressed that it is difficult to find 
resources with the “tremendous amount of images” and suggested improving the subject-specific 
searchability as well as the standard searchability (Figures 2 and 9). One curriculum coordinator 
explained that it would be helpful to search the SLL based on topics, explaining that “if you already have 
some things done based upon topic or era that teachers can easily access, then that’s going to make you 
more attractive so teachers may be willing to use the platform” (Figure 5). As for specific issues with 
building collections on the SLL website, participants mentioned both student logins and linking outside 
resources as problematic (Figures 12 and 13). Two participants expressed issues with linking outside 
resources in their collections. One of these participants said that they went back to use a collection they 
had previously developed and found that the link to an outside resource did not work anymore and thus 
they could not use the collection. The other of these participants explained that the way the site kicks out 
to other websites is problematic for student use because students then view the entire webpage the site has 
kicked out to and not the specific part of the webpage that the collection creator wanted the student to 
view. This participant said that in order to avoid this, s/he pulls out the article separately and takes out the 
part s/he wants the students to see, levels the reading for grade level, and then adds it individually to the 
collection. Ultimately, this participant wants to “minimize [student] clicks around” the SLL. Lastly, the 
social studies facilitator expressed that it is cumbersome for students to create a login and would be 
helpful if students did not have to login to interact with collections. 


pak 


Reading Recommendation: Rudolfo Anaya's “Bless Me, 
Ultima’ 
Recommendation: Have students read “Bless Me, Ultima" for a deeper 


understanding of Rudolfo Anaya, Gaspar Enriquez, and the Chicano experience in 
the American southwest in the mid 20th century. 


Type: Website 
Source: 

URL: https://book 
d=a28MRqD 
KQGAEIKTAA#HV=0 ef&q=bless% 


&dq=bless+me+ultima&hl=en&sa=X6ved=Oahl 
ne%20ultima&f=false 


Contributor: Philippa Rappoport 


Figure 12. Example of an outside resource on the Smithsonian Learning Lab 


Search the Smithsonian Learning Lab LS) *& 


Login or Sign Up 


Using Learning Lab Credentials Social Login 


alt Adares: f Facebook 


g 


_] Remember Me Forgot Password? 


Figure 13. Example of the login page on the Smithsonian Learning Lab 


Participants were asked if there was any ethnic studies content that would be beneficial to them if added 
to the SLL. It is important to note that some participants explained that they had not searched the SLL for 
ethnic studies content thoroughly prior to answering this question, meaning that some of content 
suggested may already be available on the SLL. Multiple participants explained that Texas’s State Board 
of Education approved an African American studies course this year and that curriculum specialists and 
teachers will need content regarding African American history for the upcoming academic year. When 
asked what specific African American studies content they would suggest be added to the SLL, one 
curriculum coordinator said, “a combination of local and national movements,” while another asked for 
Texas-specific content, saying “that would be beneficial, local Texas history, but even just if you have 
affiliates in other parts of the state [collect local Texas history] ... so that there’s more extensive Texas- 
specific history collections.” A few participants suggested adding other local resources as well, such as 
resources about “Round Rock,” a town that is near Austin that has a unique history. However, participants 
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questioned if these local resources are the responsibility of the Smithsonian, with one explaining, “I don’t 
think it would be the Austin centers’ responsibility to curate things on our local community but it would 
be nice if it was somewhere to get to easily.” Another participant, the social studies facilitator, suggested 
adding content about Dentin, a city with an interesting history of reconstruction. Other participants 
mentioned adding more indigenous history, Mexican American history, Asian American history, and 
women’s history. Lastly, one curriculum coordinator did not signify any specific ethnic studies content 
but did ask that affiliate collections be easier to find. For example, s/he stated “I know that each of the 
regional centers here in Austin created decks but I wish that there was a way...I’ve looked for their decks 
but haven’t seen them...I’d like to see a place on their website or links on the overall city of Austin’s 
website. Some kind of way so that it’s there and available.” 


When asked about the compatibility of the SLL with online learning platforms, the majority of 
participants did not recall finding any problems. Participants stated that they use Canvas and Google 
Classroom. Of the participants that use Canvas, one said that s/he did not have a problem using the SLL 
on the platform while another said that s/he did, stating “we haven’t successfully been able to embed in 
our canvas. It may be more of a district setting because our people limit what we can and can’t do.” Of 
the participants that use Google Classroom, only the US history teacher had used the SLL with the 
platform, explaining “when we went to virtual learning because of the pandemic, I did use it with Google 
Classroom with no problems.” The curriculum support specialist expressed the importance of a 
compatible platform, stating, “that’s something that’s very important - that your share features and links 
have to be compatible and from what I’ve seen they are. They’ ve got to be viewable on iPads... and Macs 
and once in a while we may have windows and we need to make sure it's viewable with any device and it 
is so far. That’s a stopper on some resource when it’s not compatible.” 


Suggested Ideas for Additional Smithsonian Support 


Many of the participants felt that the Smithsonian could provide more virtual support for specialists and 
teachers. They suggested a couple of ways this could be done, including live video conferences, recorded 
online trainings, and/or calls with teams of teachers who are interested. Some of the participants shared 
ideas for the content of virtual support, such as discussing “some of the new updates and things that 
people may not know about the program” and “maybe if they recorded it like, ‘this is how you make this 
lab’ or ‘this is a conversation about this and here are some decks already made.’” Three participants 
suggested that the Smithsonian provide more in-person support by conducting more workshops and/or 
meetings and attending conferences, with one curriculum coordinator saying, “I think the main thing is 
opening a broader base of workshops to get it out there to more teachers.” Another curriculum 
coordinator suggested providing regular check-ins with participants, stating, “I think regular check-ins 
and letting us know when there are updates and things that may be interests.” 


When asked specifically about local support, such as support from affiliate museums, all of the 
participants said that they would be interested in an occasional meeting of the group to share and develop 
skills. Participants explained that it would be helpful to share skills about aligning state standards to 
content on the SLL in these meetings. A couple of participants also expressed an interest in inviting 
colleagues to these virtual meetings, including other curriculum specialists and/or teachers, if allowed. 


When asked about joining a listserv to share and receive content regarding the SLL, all of the participants 
said that they would be interested. Only one of the participants elaborated on what s/he would like the 
listserv to contain, explaining that “it would be a good opportunity for the Smithsonian to roll out some 
things to people that are participating... maybe occasionally have the Smithsonian share out specific 
things that we could then use in the classroom or in my case share with my teachers.” 
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Best Aspects of the Workshop 


Participants expressed that the workshop prepared them to use the Smithsonian Learning Lab with 
teachers in a variety of ways. Over half of the participants explained that the time allotted to creating 
collections during the workshop best equipped them to introduce the website and its features to teachers. 
For example, the social studies facilitator said the aspect of the SLL that most prepared him/her to share it 
with teachers was “that ability to go through with it, do it, build it, share it — I was sold. That time to dig 
around and build and look at collections.” A couple of participants mentioned how the workshop 
introduced them the SLL website and how to navigate it, with one saying “we got a good overview of 
how it works and its purpose and intent.” A couple of participants explained that holding the workshop at 
local affiliate sites best prepared them to then share the SLL with teachers because “being able to go to 
various cultural sites around Austin and hear from people there about what’s available face to face for 
students [helps] stimulate ideas to put in your own collections when using the Learning Lab.” Two 
participants stated that the workshop thoroughly prepared them to introduce the SLL to teachers, with one 
saying “I think the best preparation is preparing us to introduce it to teachers.” The curriculum support 
specialist expressed that the workshop helped him/her contemplate how to align content on the SLL to 
standards, explaining, “in my mind, I was already aligning what I’d be looking for because I knew Texas 
state standards for ethnic studies in our state so I kind of knew what was required and what to look for.” 


Some other aspects of the workshop that participants appreciated were the following: the deadline for 
NCSS was helpful in pushing them to create collections, the theme of ethnic studies helped them focus on 
what to create, and the “session’s themselves were clear, focused, the right amount of time, [and] had 
good follow up.” A couple of participants expressed what a “wonderful job” Ashley and Philippa did at 
introducing the SLL to them, with one stating, “I think having two really great presenters that were 
personally passionate about what they’re doing helps a lot. You can tell they enjoy what they’re doing,” 
and another saying, “Ashley did a great job showing us around and how to access and create a collection.” 
Another participant mentioned how much s/he enjoys staying in touch with Ashley and Philippa and how 
s/he appreciates that they’ve reached out and made sure to keep working on the partnership, saying “it’s 
awesome to be able to connect to people doing this work.” 


Suggested Workshop Improvements 


Participants suggested a few ways that the workshop could be improved, primarily focusing on allotting 
more time to introducing the SLL website. For example, some participants expressed that time spent on 
introducing the SLL during the workshop was “rushed” and that they would have appreciated more time 
spent on explaining how to use the SLL to improve curriculum and/or how to use the SLL’s tools. Three 
participants suggested encouraging participants to create something that they could use the next day in the 
classroom, such as a collection connecting to standards and aligned with the scope and sequence teachers 
are currently using. One of these participants said, “let’s leave with something our teachers can do the 
next day.” Two participants expressed that the logistics of space could be improved, stating that it was a 
little cramped. One of these participants suggested using the spaces in the following ways: “the Mexican 
American place in Austin had more space, the labs, that was a great space for the second day so I guess 
just flipping the Asian location to be a day where we learn and do things, the African American where 
you do a meet and greet and sit in the foyer and then the Mexican American studies were perfect for the 
last day with the lab so maybe reusing the space was the only thing because it was difficult.” Two 
participants explained that providing a handout about the SLL would have been helpful for them. Lastly, 
one participant mentioned that conducting a workshop during the week rather than the weekend might 
attract additional participants. 
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It is important to note that some participants did not have suggestions for improvement at all and felt that 
the workshop ran smoothly, with one saying “I don’t know that I have any suggestions for improvement. 
Like, what we did was a day and I felt like that was the right amount of time and covered the bases. There 
was a lot of variety of activities.” 


Participants had a varied reaction to future workshops containing both curriculum specialists and teachers 
as participants. For example, the social studies facilitator felt that “they’d be great for both — it’d be neat 
to go with your own teachers and have that experience where we all go through it together,” while the US 
history teacher stated “I imagine maybe it would be different workshops for them because they would be 
developing stuff whereas teachers would be learning how to utilize things already developed. It seems 
like they would have different objectives.” 


Participants discussed a variety of ways that the future workshops could be improved with curriculum 
specialists and teachers as participants, with a few expressing that it would be helpful to provide breakout 
sessions for both types of participants. For example, one curriculum coordinator suggested, “if you’re 
going to have an all-group session like that, you’re going to have to consider what kind of breakout 
session would you have for each level of audience because what a teacher needs isn’t what I need because 
I’m not touching students like that. I’m influencing what happens but I don’t need the same introduction 
that a teacher would need. It may be that teachers have a longer opportunity to engage in it so it has to be 
tailored for the audience when you bring groups together. You can lose participants if they feel it isn’t 
relevant to the work they do,” while another curriculum coordinator suggested, “I think you start 
everybody out together and then for those curriculum specialists or coordinators, you move them off and 
say this is how, as a district, you can put this out there and the teachers could go off and differentiate 
between subject or grade level and bring them together to create things beneficial for their classroom so 
you can walk away with this collection right now and immediately take it back to their... so maybe ask 
them to come in with ideas for upcoming units and create collections that will enhance that unit.” Two 
participants explained that providing an opportunity for participants to share what they’ve created would 
be helpful - either during the workshop or after the workshop through an online conference system. One 
curriculum coordinator suggested that it may be important to both explain to teachers what an effective 
lesson on the SLL would contain (i.e. primary and secondary resources along with scaffolding) and how 
to facilitate conversations around ethnic studies content with students. Lastly, one curriculum coordinator 
suggested inviting curriculum developers from education service centers for future workshops in addition 
to public school participants. 


2) 


Primary Conclusions 
The main conclusions of this study were: 


Curricula developers who participated in professional development workshops one year earlier and 
the teachers to whom they presented the Smithsonian learning Lab valued the Lab mostly for the 
variety of authoritative and vetted Smithsonian digital resources. They envisioned the potentiality 
of the use of the Lab to engage students in inquiry-based learning. 


One of the main components of an effective professional development workshop is to include ample 
time to practice using the Lab. Our audience welcomed continuous support. 


1) Participants envisioned using the Smithsonian Learning Lab in a variety of ways. Over half of the 
participants described that they envisioned the SLL as a resource for student use. The following 
were some ideas for student use of the SLL: 


Envisioned Student Use of the SLL 


As a way for students to share their own learning 

As a way for students to create dialogue between their peers 

As way for students to conduct document-based question writing 

“As a way to move more towards the student-centered approach opposed to direct teaching.” 


“Like a resource for my students... what I like about it is the collections that are 
curated by museum professionals that are on there and those resources, those 
photographs, those collections. Those are really helpful for my kids.” 


2) Participants primarily used the SLL to develop curriculum for use based on student-centered 
approaches. They pointed out that the use can foster sharing and inquiries. Outside of curriculum 
development, participants used the SLL to find resources for content development and/or for 
personal research. Participants found the SLL to be effective for a variety of reasons, primarily 
being the abundance of vetted and reputable resources and/or the ability to use these resources to 
create content. Participants had not yet used the SLL in classrooms. 


“It is something that’s pretty much always within reach for me when a teacher says I 
need something or if we're designing something together, I’m always going to swing 
by the Learning Lab to find what's available to use. I use it a lot.” 


3) Participants described introducing the SLL to teachers through professional learning 
communities, professional development sessions, other types of meetings, and/or organic 
conversations. Participants did not know if the teachers whom they introduced to the SLL had 
used it in the classroom with their students. They did however recall that teachers expressed an 
interest in doing so and used a range of positive terms to describe the SLL. Teachers expressed 
that the SLL is effective because of the abundance of reliable resources, the accessibility of the 
website for students, and the usability of tools, such as hotspots, discussion questions, and 
annotations. 


“Whenever I have conversations with teachers looking for how to find an image that 
would be something kids could engage with in the classroom, I always mention the 
Learning Lab... because of the reputation and the fact that there’s so many, 1000s, of 
digital images that the kids would benefit from that they may never see in person.” 
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4) 


5) 


6) 


7) 


8) 


Participants had mixed opinions about teachers using the SLL to align content to scope and 
sequence. The majority of participants agreed that the SLL can be used to align content through a 
connection to standards and/or the use of primary resources in creating content. The rest of the 
participants, who did not express that the SLL could not be used to align content to scope and 
sequence, pointed out that the platform is not a necessary tool to do so. One participant explained 
that the SLL does not contain Texas-specific standards, and is thus unhelpful in aligning content 
to standards. 


“Tt allows them [teachers/curriculum specialists] to look at a standard and consider 
all of the possible way that’s a standard can be taught.” 


Participants suggested finding ways to introduce the SLL to more teachers in order to encourage 
them to create effective collections. Some suggestions for this were inviting teachers to 
workshops, providing virtual training via YouTube, sending out a listserv to teachers who are 
interested, attending state-wide conferences, and promoting the SLL through social media. 


“T think it starts with the workshop.” 


All of the participants were interested in joining an occasional meeting of a local support group 
provided by affiliate museums and a listserv to share and receive content regarding the SLL. 
Participants also expressed interest in additional virtual support. Some suggestions for virtual 
support included live video conferences, calls with teams of teachers/curriculum specialists who 
are interested, and/or recorded online training. Some suggestions for the content of virtual support 
include providing information about SLL updates, introductions to the SLL website, explanations 
of particular collections, and/or how to create content. 


“Maybe if [SI] recorded it like, this is how you make this lab or this is a conversation 
about this and here are some decks already made - anything that’s recorded and 
could be shared with teachers.” 


Participants expressed a couple of ways that the workshop best prepared them to use the SLL, 
primarily being the thorough introduction to the SLL and time allotted for collection creation. 
Participants also mentioned how effective Ashley and Philippa were at both introducing the SLL 
during the workshop and keeping in touch after. 


“That ability to go through with it, do it, build it, share it — I was sold. That time to 
dig around and build and look at collections and I immediately started talking to 
people back on campuses. The actual interaction with the lab.” 


Participants explained that the workshops could primarily be improved by allotting more time to 
introducing and using the SLL to create content, with some expressing that this portion of the 
workshop was “rushed.” As for future workshops with curriculum specialists and teachers, 
participants suggested providing breakout sessions for both types of participants to discuss their 
future use of the SLL. 


“IT would ’ve liked more time to do some of the work. I felt like touring all of the 


centers was important, but at times I felt maybe it was repetitive and while it was 
quality, it wasn’t what I thought I was signing up for.” 
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9) Participants suggested the following improvements to the SLL website: 


Suggested Improvements to the SLL Website 

Adding PowerPoint capability to the collections 

Adding an ability to “level” (i.e. change the reading level to the intended grade level) historical 
readings based on grade level 

Adding video upload capability for students who are uncomfortable typing out answers 
Creating a variety of lesson plans developed by museum educators for teacher use 
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Appendix 
Smithsonian Ethnic Studies Workshops Pre-Program Evaluation Interview Template, 2020 


A series of not more than ten in-depth phone interviews will be conducted with participants in the FY 
2019 workshops who used the Smithsonian Learning Lab with their school districts in the academic year 
2019/2020. The aim of the interviews will be, but not limited, to understand the experience, barriers, and 
lessons learned. The protocol for the interviews will be developed by the contractor in collaboration with 
SI and Affiliates staff members. The evaluation protocol will be submitted to the SI IRB. A final written 
report will be submitted by August 14, 2020. 


Interview Script 


Use of the Learning Lab with Teachers 


Thanks so much for taking the time to talk to me. We contacted you as you participated in last year’s 
workshop about Ethnic Studies and the Smithsonian Learning Lab and you used the Lab with your 
teachers during this past academic year. We are eager to hear about your experience with the Lab and 
the teachers. Please be candid as the interviews are anonymous and we'll use the analysis of the data 
only as aggregates. You may stop the interview at any time without any penalties. May I continue ? If so, I 
will be taking notes of what you say, so I will appreciate it if you could talk not too fast. 


First, let’s talk about you for a moment. How do you define your role in schools? Curriculum specialist? 
Teacher? Both? For what grades do you usually write curricula and for what subject? 


What do you teach and for what grade? 
Now, I just would like to confirm that you used the Lab with your teachers during this academic year? 
IF THEY HAVE USED THE LAB WITH TEACHERS: 


First, let’s talk about your experience with the Lab. After that, we’ll talk about your experience with the 
teachers when you shared the Lab. 


1. You participated in the workshop. FOR CS: As a curriculum specialist, what did you envision the 
Learning Lab to be used for? Did you use it to develop curricula? If so, can you please describe what 
you developed? 


2. Did you use the Lab in any other ways than developing curricula? If so, can you please describe 
them? (If the interviewee talks about sharing with teachers, then remind her/him that we are going to 
talk specifically about this topic later) 


3. FOR CS WHO ALSO TEACH AND FOR TEACHERS ONLY: Did you use the lab in your 
classroom? If so, can you please describe that experience? What were the benefits? Were there any 


challenges?) 


4. Did you find anything particularly effective about the Lab? If so, can you please describe what? (This 
could be about their teaching skills and/or about their experiences with the students) 


5. What motivates you to use the Learning Lab? How does using the lab help you help your teachers, 
particularly in aligning content to their scope and sequence? 
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6. 


When you aren’t using the Learning Lab, what are other online learning platforms and tools you find 
effective? Why do you find those platforms or tools effective? How do they compare to the Learning 
Lab? (Follow up: Do you have any suggestions about how we could add the effectiveness of those 
tools to the Learning Lab?) 


FOR CS: Now, let’s talk about your experience with the teachers and the Lab. 


ve 


10. 


11. 


12. 


13. 


14. 


15. 


Can you please describe how you’ve used the Lab with the teachers? (how many times, how many 
teachers, during a training dedicated to the Lab or... in the classroom with the students? 


How did you initially describe the Learning Lab to the teachers? Can you please describe when you 
shared the Lab with the teachers? How was this done? (This could have come up by answering Q6. If 
so: you talked about how you shared the Lab with the teachers, would you like to add anything else?) 


How do you think teachers felt about the Learning Lab? Do you know if any of the teachers created 
collections? Did they use the Lab in the classroom? If you created a curriculum with the Lab, did the 
teachers use it? If so, please describe it. 


Did the teachers mention anything particularly effective about the Lab? If so, can you please describe 
what? (This could be about their teaching skills and/or about their experiences with the students) 


Do you know if the teachers used the Lab with their students? If so, did they describe any benefits to 
you? What, if any, barriers have you found while using the Learning Lab? (If they already spoke 
about when asking question nine, then... you already spoke about using the lab with your students, 
would you like to add anything else? What, if any, barriers have you found while using the lab?) 


Did teachers have any suggestions for improvements? If so, can you please describe them? Did you or 
any teachers you worked with have any problem using the Learning Lab on online learning platforms 
such as Canvas, Blackboard, Google Classroom, etc.? 


How could the Smithsonian provide support outside of the workshops? (Follow up: As for local 
support, would you be interested in an occasional meeting of the group to develop and share skills? 
Would you be interested in joining a listserv with other participants?) 


How do you think the Smithsonian could encourage more teachers to create thoughtful and effective 
content on the Learning Lab? What do you consider effective content? 


Is there any specific ethnic studies content either from the Smithsonian or your local affiliate that you 
feel would be beneficial to add to the Learning Lab? If yes, please describe this. 


I would like for you to think about how you have used the Learning Lab with teachers over the past year 
while you answer the following questions. We are trying to better understand the long-term impact of the 
workshops on participants. 


16. 


17. 


How do you think the workshops best prepared you to use the Learning Lab with teachers? 


How do you think the workshops could be improved? Is there anything that you wish the workshops 
had covered that they did not? 


Now, I would like to speak with you about future workshops. 
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18. Future workshops will have teams of curriculum developers and teachers as participants. How do you 
think the Smithsonian could improve the workshops to be beneficial for both participants? 


19. Any final thoughts? 


IF THEY HAVE NOT USED THE LAB WITH TEACHERS: 


Let’s start by talking about how you have used the Lab. 

1. You participated in the workshop. As a curriculum developer, what did you envision the Learning 
Lab to be used for? Did you use it to develop curricula? If so, can you please describe what you 
developed? 


2. Did you use the Lab in any other ways than developing curricula? If so, can you please describe? 


3. What motivates you to use the Learning Lab? Does using the lab help you help your teachers, 
particularly in aligning content to their scope and sequence? 


4. Did you use the lab in your classroom? If so, can you please describe that experience? What were the 
benefits? Were there any challenges? 


5. Did you find anything particularly effective about it? If so, can you please describe? (This could be 
about their teaching skills and/or about their experiences with the students) 


6. When you aren’t using the Learning Lab, what are other online learning platforms and tools you find 
effective? Why do you find those platforms or tools effective? (Follow up: Do you have any 
suggestions or ideas about how we could add the effectiveness of those tools to the Learning Lab?) 

7. Can you share with me what kept you from sharing the Lab with teachers? 

I would like for you to think about how you have used the Learning Lab over the past year while you 

answer the following questions. We are trying to better understand the long-term impact of the workshops 

on participants. 


8. How do you think the workshops best prepared you to use the Learning Lab? 


9. How do you think the workshops could be improved? Is there anything that you wish the workshops 
had covered that they did not? 


Now, I would like to speak with you about future workshops. 


10. Future workshops will have teams of curriculum developers and teachers as participants. How do you 
think the Smithsonian could improve the workshops to be beneficial for both participants? 


11. Any final thoughts? 
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